Skip to content

guest-agent: fix v1 leftovers and pin the two untested v1 methods - #1118

Merged
kvinwang merged 6 commits into
nextfrom
fix/v1-audit-followups
Aug 25, 2026
Merged

guest-agent: fix v1 leftovers and pin the two untested v1 methods#1118
kvinwang merged 6 commits into
nextfrom
fix/v1-audit-followups

Conversation

@kvinwang

Copy link
Copy Markdown
Collaborator

Follow-up to #1116. A post-merge audit of the whole v1 surface — agent, proto, spec, and the four SDKs — turned up five more comments of the species that PR was already cleaning up, three documentation gaps, and two of the six v1 methods with no handler-level test. This is the agent/proto/spec half; the SDK half is a separate PR.

Comments naming things that do not exist

GetAttestationForAppKey's doc still credited "the v1 method" with sharing its report data, and the test pinning its vectors still said WorkerV1 wraps those bytes in an attestation. v1 ships neither: there is deliberately no v1 AttestAppKey, as agent_rpc_v1.proto and the spec both say at length two paragraphs earlier.

Four more places wrote WorkerV1 as a service name. The service is Worker in package dstack.guest.v1 — the proto is explicit that the names carry no version suffix, and the spec says to write "the v1 Worker". The Rust import aliases WorkerV1Client/WorkerV1Server are untouched: those disambiguate two same-named generated types in one scope, which is the reason they exist.

AttestGpu's proto comment contrasted itself against GpuInfo, a method that exists on no surface — it was removed from the unversioned surface as a never-released addition, and rpc_service_v1.rs asserts it is absent from both. The real contrast is Attest's boottime_gpu_evidence, which is the thing that returns a record written at boot.

Documentation gaps

The spec pointed at rpc_service_v1/keys.rs for the private-key vectors and for a_v0_claim_cannot_be_crafted_into_a_v1_claim. Both live in ra-tls/src/api_v1.rs; keys.rs pins only the public-key column and says so itself. A reader following either pointer to check the normative bytes landed where they are not.

The Errors table omitted the two validations AttestGpu actually performs (nonce length, and nvattest being absent from the image), while listing the other four.

Health's fail-open for an app that never opted into gating — healthy: true meaning "nobody asked me to know", not "checked and fine" — was pinned only by a test, and appeared in none of the three documents that describe the method. It is now in the proto comment, the spec, and app-health-checks.md, each saying what a non-gateway caller should read that true as.

Tests

IssueCert and AttestGpu were the two v1 methods with no handler-level coverage, so nothing would have noticed the v1 handlers ceasing to route through the shared validity check or through the attestor. Both are now asserted through the handler rather than against the helper in isolation, and the on-demand GPU format tag is pinned like its boot-time counterpart already was — a consumer selects its verifier on (vendor, format), so both tags are wire contract.

The AttestGpu test formats the error with {:#} deliberately: the handler adds a context line, and ra_rpc encodes errors as format!("{error:#}"), so asserting on the flat to_string() would test something no client ever sees.

Verification

  • cargo test -p dstack-guest-agent --all-features: 118 passed (3 new).
  • dstack-gateway: 289 passed. frozen_surface: 2 passed — the frozen descriptor digest is unmoved, as a comment-only proto edit requires.
  • cargo fmt --all --check and cargo clippy -D warnings (per CLAUDE.md) clean.

A post-merge audit of the v1 surface turned up five more comments in the
species #1116 was already cleaning up, three documentation gaps, and two of
the six v1 methods with no handler-level test.

Comments naming things that do not exist: `GetAttestationForAppKey`'s doc
still credited "the v1 method" with sharing its report data, and its vector
test still said `WorkerV1` wraps those bytes in an attestation -- v1 ships
neither. Four more spots wrote `WorkerV1` as a service name; the service is
`Worker` in package `dstack.guest.v1`, and the proto says so explicitly.
`AttestGpu`'s proto comment contrasted itself against `GpuInfo`, a method
that exists on no surface: the real contrast is `Attest`'s
`boottime_gpu_evidence`, which is the thing that returns a boot record.

Documentation gaps: the spec pointed at `rpc_service_v1/keys.rs` for the
private-key vectors and the v0-forgery regression test, both of which live
in `ra-tls/src/api_v1.rs` -- a reader checking the normative bytes landed
where they are not. The Errors table omitted the two validations `AttestGpu`
performs. And the `Health` fail-open for an app that never opted into
gating -- `healthy: true` meaning "nobody asked me to know" -- was pinned
only by a test, in none of the three documents that describe the method.

Tests: `IssueCert` and `AttestGpu` had no handler-level coverage, so nothing
would have noticed the v1 handlers ceasing to route through the shared
validity check or the attestor. Both are now asserted through the handler,
and the on-demand GPU format tag is pinned like its boot-time counterpart.
Copilot AI lite review requested due to automatic review settings August 24, 2026 16:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Comment thread dstack/guest-agent/src/rpc_service_v1.rs Fixed
…nature

The comment said a malleated signature "fails to parse rather than
verifying". Measured against k256 0.13.4: `Signature::from_slice` accepts the
high-S bytes -- it only rejects an `r` or `s` outside `1..n`, and `n - s` is
still in range -- and the rejection happens inside verification. So a caller
gets HTTP 200 with `valid: false`, not the 400 a parse failure produces.

The security answer is unchanged and the behaviour is right; only the
explanation was wrong, and it was wrong about the status code, which is the
part a client branches on.
… 400

The doc comment on the frozen `Verify` makes an observable claim: k256 rejects
high-S inside the verification rather than in `from_slice`, so a malleated
signature comes back as HTTP 200 with `valid: false`, not as the 400 a parse
failure would produce. Nothing pinned it.

The test builds the malleation by negating `s`, then checks it is the real
thing before using it -- `normalize_s` is `Some` only for a high-S signature,
and it must normalise back to exactly the signature that just verified -- so
the assertions cannot pass against a no-op. Both halves of the claim are
asserted: `.expect()` for the status code a 0.5.x client branches on, `!valid`
for the security answer.

A k256 upgrade that moved the check into parsing would keep the second and
silently break the first. That is the drift the comment warns about.
Three edits from review of the preceding two commits.

`GetAttestationForAppKey`'s comment said the report data was "the same
commitment the frozen `Sign` path's key uses". `Sign` uses a key, not a
commitment. Say what is actually shared: the derivation -- same path, purpose
and base algorithm -- so the attested public key is the one that signs.

The Errors table now lists a row that no client can clear by changing its
request. Since a 400 in this API means "the method ran and failed" rather than
"your request was malformed", say so where the table says everything above is
a 400, and say that the message in the body is what separates the two.

`app-health-checks.md` restated the fail-open mechanism the proto comment and
the spec already carry. Keep the reading an operator needs -- what `true` means
for an app that opted out -- and drop the third copy of how it is computed.
Also reflow the paragraph the `ra-tls` pointer fix left short of the fill.
… GPU

`AttestGpu` reported "GPU attestation is not available in this image" as an
uncoded error, which `dispatch_prpc` turns into the generic 400. That tells a
client its request was malformed. It was not: the request is well-formed, and
no other request would succeed either, because the image ships no nvattest and
will not grow one at runtime. A client branching on the status retries with
different arguments forever instead of falling back.

`ra_rpc::ErrorExt::with_code` already carries a chosen status through the
transport, and `code_of` walks the whole error chain, so the code survives the
handler's `.context("GPU attestation failed")`. 501 rather than 503 because the
capability is absent for the lifetime of the CVM, not temporarily unavailable.

A malformed nonce keeps the default 400 -- that one really is the caller's
fault, and the two failures must not be indistinguishable.

Safe to change now: `AttestGpu` is v1-only and never shipped in a release, so
no deployed client is reading the old 400. `rpc_service_v1.rs` already asserts
it is absent from both frozen surfaces.

The availability probe moves from `nvattest::available()` to a `GpuAttestor`
field holding the binary's path. Without that, a test for the unavailable
answer would pass only on a host with no nvattest installed and would spawn a
real collection against the host's GPUs anywhere else. The test fixture pins it
to a path that cannot exist, so every guest-agent test sees the same answer.

Documented in the proto comment, the spec's Errors and Status codes sections,
and the curl API reference, which listed only 400 and 500.
`return Err(anyhow!("...").with_code(501))` is four moving parts to say one
thing, and it reads as an error construction rather than as a control-flow
exit -- the two existing sites both wrote it differently, one importing
`ErrorExt` and one calling it as a free function.

`ra_rpc::bail!(501, "...")` expands to exactly that expression. The format
arguments are forwarded to `anyhow::anyhow!` verbatim, so nothing new has to be
learned, and the macro resolves `anyhow` through a hidden re-export so a caller
does not need it in scope.

The code is the first argument and is mandatory. An error with no status to
choose should keep using `anyhow::bail!` and be reported as CODE_BAD_REQUEST,
which is the correct default; requiring the code here means the two spellings
say which kind of failure they are. `gpu_attest.rs` now shows both in one
function -- a bad nonce bails plainly, a missing nvattest bails with 501.

Converted both existing sites. The 413 path in `read_data` is already covered
end to end by `status_codes::a_payload_over_the_limit_is_rejected`, so the
macro reaching the transport is pinned without a new test; the unit test added
here covers the part that test cannot see, that the code survives the
`.context(..)` a handler adds on the way out.
@kvinwang
kvinwang enabled auto-merge August 25, 2026 02:02
@kvinwang
kvinwang merged commit 366317b into next Aug 25, 2026
16 checks passed
@kvinwang
kvinwang deleted the fix/v1-audit-followups branch August 25, 2026 02:08
kvinwang added a commit that referenced this pull request Aug 25, 2026
`AttestGpu` on an image that ships no nvattest answers 501, not 400 --
the request is well-formed and no retry of it will ever succeed, so a
client that reads 4xx retries forever while one that reads 501 falls
back. The agent started saying so in #1118; these tests still asserted
the old 400, and the JS one failed against a `next` that had moved.

Widened rather than relaxed. The JS assertion now names 501 and the
agent's own words instead of matching any 4xx, and the Rust and Python
tests -- which only checked that *something* failed -- pin the status
too. That is this branch's own claim under test: every SDK reports a
non-2xx response with both the server's error text and the HTTP status,
and until now nothing proved it for the one status that tells a caller
to stop trying.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants